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ABSTRACT 



This paper is in response to a growing concern about the adequacy 
of present measures and methodologies in depicting air-to-ground delivery 
system accuracy. An accuracy measure, measurement methodology and 
performance guarantee based on cost-effectiveness techniques are 
developed. The measure is based on the frequency of mission success. 

The methodology is based on Bayesian techniques using a multinomial 
distribution to represent the radial miss distance pattern. A technique 
for using an estimate of CEP to determine the prior parameters is 
developed. Actual and simulated impact data are used to compare the 
proposed methodology to historically accepted and other recently proposed 
techniques. The methods of cost-effectiveness are applied to guaranteeing 
an air-to-ground system in an effort toward making costs more controllable. 
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I. INTRODUCTION 



A. PURPOSE 

The intent of this thesis is threefold; to propose a measure of 
effectiveness; to propose a method to determine the specific effective- 
ness; and to propose the concept of a cost-effectiveness guarantee. 

All three are proposed with respect to an air-to-ground (A/6) weapons 
delivery system in a dive maneuver. However, the techniques proposed, 
with the necessary modifications, appear to have wider applications. 

Some of the development is incomplete and requires further investigation. 



B. NEED FOR ACCURACY MEASUREMENT 

Prior to developing a method for measuring the accuracy of an A/6 
delivery system, it is important to understand the need for accuracy 
measurement. Several needs are discussed. 

1 . Specification Compliance 

In the Test and Evaluation (T&E) community a determination of 
specification compliance is usually the prime motive for measuring 
accuracy. The accuracy measure and occasionally methodology are speci- 
fied in the applicable contract guarantees. Confusion can be introduced 
by the use of different measurement units and methodologies in different 
specifications. 

A typical, though fictitious, example of an accuracy specification 
is: 

"Exhibit a 15 mil circular error probableCCEP) , corrected f9^^ 
aiming error, computed in the plane normal to the line of sight 
from the release point when delivering Mk-76 practice bombs 
at a release airspeed between 400 and 450 KIAS, a dive angle 
, between 40 and 50 degrees and a release slant range to target 
less than 10,000 feet." 

Testing to this specification would probably be expensive. 
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Also inherent in the specification example is the singular usage 
of the measure of accuracy. This measure in itself has little value 
outside of the specification context. Therefore, it appears that a 
significant amount of money and time would be expended with negligible 
information gain beyond specification compliance. 

2. Sortie Predictions 

In the employment of an A/G system there is a need for a differ- 
ent measure of accuracy. Strike planners and weaponeers need to estimate 
the probability of a weapon impacting within a given distance of a target. 
(The methodology used for this purpose is described later.) This is an 
important need as it is required throughout the lifetime of the system. 

3, System Comparisons 

Often a need arises to compare a system with some other system 
or systems. One of the measures of effectiveness that should be used for 
comparison is accuracy. Thus, a need arises for a measure of accuracy, 
common to the various systems being compared. An identical need exists 
if a cost-effectiveness analysis is to be conducted. 

Three general requirements for measurement of accuracy have been 
discussed. As presented, each of these requirements uses a different 
measure and the measures need not be compatibly defined. In theory, it 
could occur that several separate determinations of accuracy would be 
required for a single system. In practice, this is not usual, however, 
a common technique would be beneficial so that all requirements could 
use the same measure. 

C. HISTORICAL MEASUREMENT METHODS 

Historically the measurement of accuracy of nearly all types of 
weapons delivery systems has involved probability distributions and the 
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parameters that define these distributions. Several terms that are 
commonly used in weapons accuracy analysis are shown in Figure 1. 

1 . Distributions 

The normal family of probability distributions has long been 
accepted as the proper family for the distribution of weapon impacts. 
Recently this concept has come under closer scrutiny. It has been 
suggested that a normal distribution may not always accurately depict 
a parent distribution of impacts and other families such as the Cauchy 
have been proposed [Ij. 

a. Normal and Variations 

Many different variations of the normal distribution have 
been used or proposed but only the two most common are discussed. 

(1) Bivariate Normal . The bivariate normal is the more 
flexible of the two variations discussed, permitting the errors in range 
and deflection to be correlated. Letting x denote range and y denote 
deflection, the density function is: 



(1) f(x,y) = 



1 






exp{ 



-1 



2(1-P^) 



2 

{(_^)2_2p 



x-Pv y-p, 






Often the range and deflection errors are assumed to be nearly uncorre- 
lated so that p is approximately zero. A rotation of the coordinate 
system can also be used to eliminate the correlation but the resulting 
variables do not represent true range and deflection errors. The 
uncorrelated density reduces to: 



(2) f(x,y) 



2.a^Oy 
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Figure I. 

Depiction in the Ground Plane of terms 
commonly used in Weapons Accuracy 
Analysis. 
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The parameters are: 

- mean of the range error distribution 

u - mean of the deflection error distribution 

y 

2 

- variance of the range error distribution 

a ^ - variance of the deflection error distribution 

y 

p - correlation between the range and deflection errors 

Standard statistical techniques can be used to estimate these parameters 
from observed data. 

(2) Circular Normal . The more commonly used normal distri- 
bution is the circular normal in which it is assumed that the range and 
deflection errors are independent (uncorrelated), have mean zero and 
have common variance. The circular normal cumulative function where r 
is the radial miss distance is: 

2 

(3) F(r) = 1 - exp } ; r ^ 0 

2a 



Procedures have been developed for use when f 
[2, p. 3 and 4]*, i.e., the distribution is an elliptical normal. 

An axis rotation can be used to eliminate the correlation between range 



and deflection. 



An 



estimator for the o 



2 



parameter is: 



’2 

a 



N 

I r 



2N 



2 



The circular normal has computational simplicity over 
the more general bivariate normal distribution, but the assumptions are 
quite restrictive. 
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b. Cauchy 

An Air Force report [1] of an analysis of combat impacts 
observed in Southeast Asia found that the normal family did not yield a 
good distributional fit of the data. One of the distributions that was 
shown to closely approximate the data was the Cauchy. 

The functional forms of the Cauchy were proposed; 



(4) 


Rectangular Cauchy 


F(x,y) 


= ^ tan’^ 
ir 




-1 _y. 
23 


(5) 


Circular Cauchy 


F(r) = 


— tan"^ ( ■ 

7T 




)l/2 


.2 ^ 2 



+ r 



The estimator for the parameter g was proposed as: 

A 

e = .455 • (radial miss distance of the median impact) 

The functional forms of the Cauchy proposed have the 
advantage of simplicity but lack mean values, 
c. Direct Hits Plus Distribution 

The Air Force study [1] pointed out that several impacts 

were observed to be directly on target contrary to the predictions of 

a continuous distribution function. This may be explained by considering 
the method of measuring the miss distances (photographic), the physical 
size of the target and some ego oriented, psychological attraction of 
the target itself. The method proposed to account for the positive 
mass at the target was to introduce a proportion of direct hits. The 
distribution function then becomes: 

(6) G(x,y) = 6 + (1-6) F(x,y) in the bivariate case, or 

(7) G(r) = 6 + (1-6) F(r) in the circular case. 

The parameter 6 could be estimated from prior experience or 
could possibly be some universal constant derived from many different 
systems . 
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d. Other Distributions 

Literally hundreds of different distributions such as the 
Weibul , exponential or uniform or variations of distributions such as 
a mixture of two circular normals or a localized normal have been 
proposed. Two interesting facts emerge from these proposals. First, 
as the predictive ability of the distributions increases, the complexity 
of the functional form generally increases. This usually increases 
manipulation difficulty. Secondly, nearly all of the distributions or 
variations proposed are continuous. 

2. Measures 

There are several commonly accepted measures of accuracy which 
are functionally related to the parameters defining a unique distribution 
in an assumed family of impact distributions. These measures are 
graphically portrayed in Figure 2. 
a. CEP 

The circular error probable is defined as the radius of a 
circle centered at the target (or designated point) which contains 
50 percent of the observed independent impacts, or, the radial distance 
to the median impact [4, p. 1-1, 1-2]. The functional relationship of 
CEP to the distribution parameters is found from the following integral: 
CEP 

(8) F(CEP) = / f(r) dr = .5 

0 

As an example, the CEP using the circular normal distribution is derived 
as: 
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Figure 2. 

Depiction of Accuracy Measures in 
the Ground Plane. 

© Individual impact points 
( 7 randomly placed ) 
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. 1 - exp (- £i^) = exp (- 



2a 2a 



lnC.5) = - 




CEP = a\/2 ln(2)‘ = 1.1774a 



b. REP/DEP 

The range error probable and deflection error probable are 
similar to CEP. The REP is the distance from the target (or some desig- 
nated point) to the median impact in range [4,p. 1-2,1 -3], The functional 
relationship of REP to the distributional parameters is: 

REP 

(9) F(REP) - F(-REP) = .5 = / f(x) dx where f(x) is the 

-REP 

marginal distribution of x from f(x,y) 

DEP is defined similarly. For a bivariate normal distribution, 

REP = .674a^ and DEP = .674a^. 

X y 



uncorrelate the range errors and deflection errors, the functional form 
in equation (9) is no longer valid. This is a serious disadvantage of 
mathematically uncorrelating the data, 
c. MRE 

The mean radial error is defined as the mean of the radial 
miss distance distribution. It is derived below for the circular normal 
distribution: 



It should be noted that if an axis rotation was used to 



f(r) = ^ exp {- ^ } 
a 2a 

MRE = E[r] = / ^ e 
0 



exp {- ^ } dr 
2o^ 
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this integral is solved [5,p. 458] as 



E[r] 




1 .253a 



MRE 



Of the listed measures, the MRE is probably the least , 
frequently used, 

d. MPI 

The mean point of impact is the point which has as its range 

and deflection coordinates the arithmetic means of the range and 
/ 

deflection coordinates of the individual impact points [4, p. 1-3], 

The MPI is calculated from an observed sample of impacts rather than 
a hypothetical distribution. In many analyses (such as T&E) [Refs. 18, 
20,23, and 24], the MPI is used as the center of the observed impact 
distribution instead of the target. The offset of the MPI from the target 
is termed a system bias and the CEP, REP, DEP and MRE are calculated with 
respect to the MPI. Obviously, this technique would not be useful in 
weaponeering applications. 

Analogous to the MPI are the mean point of impact in range 
and deflection (MPIR, MPID) which are the arithmetic means of the impacts 
in range and deflection, 

e. Other 

Three other definitions given in the Joint Munitions Effective- 
ness Manual (JMEM) [4, p. 1-1 ,1-2] are of interest and are quoted. 

(1) "Bombing Error . The combination of all errors which 
cause weapons to miss the target. Included are ballistic, aiming, release 
and aircraft system errors." 

(2) "Ballistic Dispersion . The variation of the path of a 
weapon which is attributed to physical tolerances in the weapon dimensions 
and aerodynamic stability." 
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(3) "Delivery Accuracy . The measure of the ability of 
pilots^ to put the weapon impact pattern center (usually MPI) on the 
target or aimpoint. The unit of measure of the variation in placement 
of the pattern center may be a, CEP, or REP and DEP. It is these measures 
that are used in predicting the results of future weapon releases of the 
same type. Delivery accuracy is based on the errors in aiming, release 
and aircraft systems. It does not include ballistic errors." 

3. Mils 

In many applications the accuracy measures are expressed in mils 
(milliradians) perpendicular to the line of sight (LOS) at release or 
some other appropriate point along the aircraft flight path. The geometry 
involved in the computation of the mil is presented in Figure 3. A mil 
is usually defined as the angle subtended by a secant line of one foot 
length at a radius of 1000 feet [4, p. 4-2]. The deflection mil error (d^) 
is related to the deflection foot error (d^) below using the symbology 
of Figure 3; 

1000 d. 

(11) d^ = ^ 

m s 

Due to the geometry, the deflection error in the ground plane is identical 
to the deflection error in the scoring plane, thus no correction is 
required. The range mil error is computed by the following equations: 

1000 r 

(12) r^j^ = ^ where r^ can be closely approximated 

(especially if s/r^ is large) by 

(13) r^ = r^ cos e 

^It seems to this author that more is involved than just the ability 
of the pilot. 
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Aircraft at release or some other designated point 



Horizontal 




6 = Depression of the Los from the 
horizontal. 

S= Slant range to the target in feet. 

r^: = Range miss distance in the ground 
plane in feet. 

= Range miss distance in the scoring 
plane in feet. 

r^= Mil error. 



18 




I 



I 



Theoretically, the use of mils transforms impact data taken 
from various release slant ranges and dive angles to a common base. 
Traditionally, separate accuracy measures are specified for each given 
set of release conditions; i.e., slant range, flight path angle, airspeed 
and maneuver. The use of mils permits some aggregation of accuracy data 
over release conditions. 

D. PROBLEM 

From the material thus far presented it is concluded that a single, 
general accuracy measure and methodology could be advantageous. Some of 
the more important attributes of a good measure and methodology are: 

a. suitability to each need for the measurement of accuracy. 

b. exactness^ of the estimation methodology. 

c. minimal amount of testing required to achieve the desired 
estimation exactness. 

d. freedom from distributional and other assumptions restricting 
the applicability. 

e. independence from delivery conditions, 

f. mathematical tractability and computational simplicity 

1) for data analysis. 

2) for weaponeer! ng. 

3) for maintenance of capability records. 

g. adapt ibility to new systems. 

Even the few attributes listed above indicate the difficulty of 
deriving an "optimal" measure and measurement methodology. Additionally, 



^Exactness is defined as the minimum error throughout this thesis. 
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the preferential ordering assigned the attributes by an individual will 
significantly influence his judgement of the "goodness" of a particular 
method . 

By carefully defining the problem and attempting to give preference 
to all phases of the problem, a methodology has been developed. This is 
not proposed as the "optimal" method but it does have more of the attri- 
butes listed above than previous methodologies known to this author. 
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II. ACCURACY MEASURE 



Prior to developing a methodology for determining accuracy, a specific 
measure of accuracy should be determined. 

A. SYSTEM FUNCTION 

The JMEM definition of delivery accuracy (quoted earlier) can be 
paraphrased as the placement of an impact pattern center on a designated 
point. The measure of accuracy being some description of the precision 
of that placement. However, accuracy of an A/G delivery system may also 
be thought of as the frequency with which a system performs its assigned 
function or functions. This definition differs from the JMEM in that it 
does not include assumptions about an impact pattern . (it is distribution 
free) and it gives the accuracy measure in terms of the mission. An A/G 
delivery system has the singular function or mission in a combat role of 
target destruction. 

Two important results can now be discerned. First, the singular 
function of target destruction greatly simplifies the development that 
follows. Second, a basis for the proposed definition of delivery accuracy 
has been reached. This basis can be built upon to derive a measure and 
methodology that are related directly to the system mission. 

B. MEASURE OF EFFECTIVENESS 

Based on a mission of target destruction, the accuracy measure 
definition is the frequency with which target destruction is achieved. 

An interpretation of the frequency could be the number of targets destroyed 
per sortie. Note that this is the inverse of a common weaponeering 
measure, the number of sorties required to destroy a target. The averaging 
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over sorties is not essential. It could have been taken over weapons 
expended, attacks or other quantities. Sorties was chosen because it 
is commonly used as a normalizing quantity in aviation terminology. 

It may seem that the number of targets destroyed per sortie is more 
a measure of effectiveness (MOE) than an accuracy measure. In many 
contexts, the MOE may be a function of an accuracy measure. For brevity, 
the term MOE will be used and should be interpreted as meaning both 
accuracy measure and measure of effectiveness. 

The MOE chosen (targets destroyed per sortie) closely resembles that 
used by the Weapon System Effectiveness Industry Advisory Committee 
(WSEIAC) [6, p. 24,25] which presents some recent analyses of related 
problems. 

C. WEAP0NEERIN6 EQUATION 

The MOE chosen can be mathematically expressed by a probability 
statement; the probability that the target is destroyed in a certain 
number of sorties. This is symbolized in the familiar concept of the 
probability of failure equals the product of the probabilities of failure 
on each of several assumed independent trails as; 

(14) a = 1 - (1 - where; 

a is the probability that the target is destroyed, 
p is the probability of target destruction in a single attack, 
n is the number of sorties 
% is the number of attacks per sortie. 

Solving for the rate of target destruction per sortie 0/n) yields: 
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(15) 1/" ' 1n(1-») 



Equation (15) will be called the "weaponeer! ng equation". 

For convenience in this thesis, the number of attacks per sortie (t) 
will be assumed to be unity. In many applications, such as against 
heavily defended targets, i is unity by policy. However, no loss of 
generality or mathematical inconsistencies are imposed due to taking l 
to be unity. 

Superficially, the probability of target destruction in a single 
attack (p) may appear to be identical to the historical measure of 
accuracy. A closer examination shows p to be more. Inherently, p 
depends on the destructive radius of the weapon/target combination 
However, the results of the WSEIAC analysis [7,p, 22-33] give a deeper 
insight into p. The WSEIAC report concluded that p should be related to 
the availability, dependability and capability of the system by the 
following equation, 

(16) p = A^DC where 

A^ is the transpose of the vector of probabilities that the 
system is in some state the start of the mission, 

D is the matrix of probabilities that the system is in 
some state at the required mission time conditioned on 
the state of the system at the start of the mission. 

C is the vector of capabilities conditioned on the system 
state. 

Appendix C gives an example of the use of the above equation to enhance 
understanding of this important concept. 

Now it can be seen that C, the capabilities, are what were commonly 
referred to historically as the measure of accuracy. An important 
distinction exists though that requires additional investigation. 
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D. CAPABILITY 

As defined, the capability is related to r^ and conditioned on the 
system state. No mention is made concerning the delivery conditions. 

This is a radical departure from the historical method of tying the 
accuracy to a specific set of delivery conditions. In theory and in 
reality, the probability of placing a weapon within a given distance of 
the target is dependent on the delivery conditions. However, the 
experienced attack aviator who has flown in combat or attempted to 
achieve a specified set of delivery conditions over an unfamiliar target 
and terrain on the first attack will readily admit the difficulty 
involved. The presence of enemy defenses affects all delivery parameters 
and can cause large deviations from programmed dive angle, airspeed, 
release altitude^ and run-in heading. Cloud conditions different from 
those predicted may dictate last minute changes in dive angle, release 
altitude and run-in heading. Winds not anticipated can cause dive angle 
and run-in heading to change during the attack. In close formation attacks 
each aircraft usually achieves a different set of delivery conditions. 

In older, manual delivery systems, the sight setting used was based on 
a precise set of delivery conditions. Thus, a great importance was 
imposed on the achievement of these prescribed conditions. Now, sophis- 
ticated systems continually compute the predicted weapon impact point 
and automatically release the weapon when some designated point coincides 
with the predicted impact point. The pilot using this system is free to 
vary his delivery conditions "within reason" as he deems appropriate. 



^Dive angle and release altitude define the slant range to the 
target which is an influential parameter in the accuracy. 
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It seems reasonable to insist that the capability measure also be 
free of delivery conditions. 

The "within reason" phrase cited above gives qualitative bounds on 
the delivery conditions. These bounds should reflect the current 
operational tactics, and can be expected to change with time. For 
example, in the "Vietnam Era," the bounds might have been: 



a. 


Dive angle 


25 to 60 degrees 


b. 


Release altitude 


4000 to 7000 feet 


c. 


Airspeed 


400 to 600 KIAS 


d. 


Run-in heading 


0 to 360 degrees 
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III. ACCURACY ESTIMATION METHODOLOGY 



The proposed MOE requires the estimation of p = A^DC. The estimation 
of availability and dependability (A and D) has been well documented in 
numerous reports including the WSEIAC [7]. A method of estimating 
capability (C), as used to estimate p, is developed herein. 

A. DERIVATION 

The detailed derivation is presented in Appendix A. The derivation 
of the estimator for C follows a Bayesian approach. The techniques of 
the Bayesian approach are explained by DeGroot [8], Savage, Raiffa, 
Schlaifer and many others. The derivation is summarized below to provide 
continuity. 

A squared error loss function was derived as: 



(17) L = 5- [ln(l-p)-ln(l -p)] where p is the estimator of p. 

(1n(l-a))^ 

The Bayes estimator^ was found to be: 

(18) D* = E[ln(l-p)] (the symbol " * " indicates a Bayes estimator) 

Members of the multinomial family of distributions were chosen as 
the sampling distributions because they can be used to approximate any 
distributional shape. The probability mass function of the multinomial 



is: 

(19) f (X"| j . . « ,Xj^ I N ,p-| , , . . ,P|^) 






1 



A Bayes estimator is the one which minimizes the expected loss. 
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where N = 



E X- 

i=l ^ 



1 = 



2 Pi 

i=l ^ 



Pi ^ i=l,...,k 



.•th 



and x^. is the number of impacts observed in the i interval 



The conjugate family for the multinomial is the Dirichlet. Its density 
function is; 



(20) f (p-| J • • • jP|^ I Ct-j J « . . — 



r(a-j+ ... + a|^) a-| -1 

r(cx^) r(a,^) 



Pk 



ak-1 



where 



1 = S: p. 

i=l ^ 



p^. > 0 i=l,...,k 



(X i > 0 i— 1 , . . . ,k 



Using the conjugate family, the parameters of the posterior Dirichlet 
distribution of the p^ are a-j + x-|,..., + Xj^. 

With an assumption and an approximation, the Bayes estimator for C was 
found to be 

^ m a. + X. k 

(21) C* = I — where a = E (’a. + x.) , and m is the 

1=1 “o . “ 1=1 ' ' 

smallest integer greater than or equal to 
r^/interval length of the multinomial 
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It was found that the prior parameters could be assigned by assuming a 
prior value for CEP and making some assumptions. 

Appendices B, C and D delineate the proposed procedure using an 
example problem. 

B. ASSUMPTIONS 

Several assumptions are made in the measure and measurement methodology 
derivation. This section will present a complete listing of all the 
assumptions and corresponding justifications. 

1 . Independent Trials 

Independent trials were assumed in the weaponeering equation 
(section II. C). In testing, the independence can be achieved by using 
only the first weapon dropped in a series or by randomization of the 
delivery maneuvers and parameters so that no two consecutive deliveries 
are the same. For prediction, the independence assumption is conservative 
as it neglects the possibility of multipass improvement. 

2. Accuracy Definition 

The "frequency with which a system performs its assigned function(s)" 
definition is one of many possible definitions. The rationale behind 
the particular choice was the direct link to the mission as discussed in 
section II. A. 

3. Accuracy Measure 

The "rate of target destruction per sortie" is, again, one of many 
possible choices. The rationale was discussed in section II. B. 

4. Single Attack per Sortie 

The assumption that i = 1 was made for convenience and has no 
effect other than simplification. Any value may be assigned to i without 
changing the derivation. 
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5. Independence from Delivery Conditions 



The specific assumptions and an extensive discussion of the 
rationale was presented in section II. D. 

6. Squared Error Loss Function 

The rationale behind the choice of the quadratic loss function 
was the generality and tractibility of that form. The implicit assumption 
of the loss being equal for overestimates and underestimates is a simplifi- 
cation. If separate treatment is preferred, the methodology is still 
correct as shown in Ref. 17, p. 195-197. 

7. Radial Miss Distance 

The radial errors were analyzed vice separate treatment of the 
range and deflection errors. The rationale being that any emphasis placed 
on a particular heading, or more importantly, heading relative to some 
target axis may lead to erroneous conclusions. Ideally, use of the proper 
run-in heading can increase the accuracy but due to the reasons cited in 
section II. E, this will often result in an overestimate of the true 
accuracy. 

The radial measure is expressed in the ground plane. The advan- 
tages of using mils in the scoring plane are obvious, especially in view 
of the varied release conditions. Unfortunately, targets are usually 
found in the ground plane and most weapons detonate on impact with the 
ground. The geometry of the problem will also show that weapons designed 
to detonate at a fixed altitude yield to a miss distance measured in a 
ground plane projected horizontally up to the burst height. 

8. Perfect Reliability 

The assumption that A^D = 1 was made for simplicity. Assuming 
otherwise complicates the mathematics but does not alter the concept. 
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9. CEP as Assigned Prior 

Assigning a CEP is one of several methods of assigning the 's 
required in the prior distribution. The rationale for choosing CEP is 
given in Appendix A but other schemes might suffice. 

10. Uniform Distribution of pj 

The justification for setting all the E[p^J equal, up to the CEP, 
is based on the resulting mathematical simplicity. Other assignments 
might be acceptable but might also add to the difficulty of analysis. 

11 . Each Pi Distributed Symmetrically 

Assuming each p^. is distributed symmetrically about its expected 
value is another simplifying step. Assignment of specific values to 
V[p^.] is possible for i=l,...,j but adds another subjective decision to 
the analysis. 

Deletion of assumptions 4 and 6 through 11 does not affect the 
methodology. The mathematics become more cumbersome and the estimator, 

D*, may differ, but the concept remains unchanged. 

Assumptions 9 through 11 simplify the determination of the a.j's and 
as will be shown in the numerical analysis [section IV), yield good 
results. However, changing these assumptions does not alter the estimator. 

C. APPLICABILITY 

Both the accuracy measure and methodology have been derived. It is 
worthwhile now to reflect back to the listed attributes (section I.D) 
and comment on the compliance with them. 

1 . Suitability to Needs 

The three needs cited were specification compliance, weaponeering, 
and system comparison. The suitability of the selected MOE to the latter 
two needs is apparent. The weaponeering measure was used as the MOE and 
an MOE is one of the essential elements of system comparison. 



30 



The sutlability to specification compliance is not obvious and 
in view of present specifications is even dubious. In section V it will 
be shown that specifications can be couched in terms of cost-effectiveness 
with many attendant advantages. The suitability should become apparent 
in that context. 

2. Exactness 

The methodology derivation results in a Bayes estimator of the 
accuracy measure. The Bayes estimator derived is a sufficient statistic 
[8, p. 159] which means that no more information relative to the estimate 
can be garnered from the data [8, p. 155]. The use of Bayes procedures 
also allows probability statements to be made about the parameter (p^) 
of interest. 

The numerical analysis (section IV) shows that the proposed 
methodology is the most exact of the several techniques compared. 

3. Minimal Testing 

The numerical analysis (section IV) shows that an average error 
in n^ of approximately three sorties per target destroyed can be achieved 
with 100 data points. For r^ values in excess of 40 feet, the corresponding 
error is less than two. The 100 impacts is less than that required to 
achieve comparable exactness with the other techniques evaluated. 

4. Distribution Free 

The multinomial density permits the data to define its own distri- 
butional shape as shown in Appendix A. 

5. Independent of Delivery Parameters 

The independence from delivery conditions was discussed in 
section II.D. 



Hhe reason for shifting to n vice 1/n is explained in section IV. 
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6. Tractibilitv and Simplicity 

The use of a conjugate family simplifies the data analysis as 
shown in Appendix B. The weaponeering can be accomplished from one 
table and one graph as shown in Appendix D. As the number of data points 
increases, the influence of the prior decreases. Thus, operational units 
which usually collect extensive data, need only maintain the total number 
of impacts in each of the appropriate intervals. 

7. Adaptibility 

The adaptability is highly dependent on the nature of the 
adaption required. No quantitative assessment of the adaptability can 
be made without knowing the specific adaptation required. 
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IV. NUMERICAL ANALYSIS 



Numerical analysis was used to compare the proposed method with 
other historical techniques. The Center for Naval Analysis graciously 
provided extensive A-7E weapons delivery data. There were 1244 impacts 
delivered from system dive maneuvers with reasonable release conditions. 
These data were used to make a comparison with real world data. Simu- 
lation data was used to make comparisons based on various known 
distributions with known parameters. A sensitivity analysis of the CEP 
value used for the prior was also conducted. 

The quantity used for comparison was the inverse of the accuracy 
measure proposed. The magnitude of the actual minus predicted 1/n would 

A 

be inversely related to the magnitude of n ( l/nn|n-n| ) and would lose 
meaning if n were not presented. In the interests of security, the 

A 

value of n will not be presented. So, A = |n-n| is the parameter 
compared. (This term. A, is referred to as the comparator in the figures.) 

A. A-7E DATA 

The accuracy parameters derived from the A-7E data are not given, again, 
in the interests of security. 

The true distribution of impacts was assumed to be that defined by the 
data. The large number of impacts (1244) lend credibility to this 
assumption. The n used in computing A was derived from the percent of 
observed impacts within the appropriate r^ value. 

The techniques compared to the proposed methodology were based on 
the commonly used circular normal distribution and on the recently proposed 
Cauchy distribution (see section I.C.l.b). Both the regular distributions 



33 



I 




and the distribution plus a percentage of direct hits were used. The 
percent of direct hits was that observed in the data. The parameters 
necessary for the comparison distributions were derived from the observed 
data using the estimators shown in Section I.C.l.a and I.C.l.b. 

A sample size of 100 randomly selected data points was chosen as 
being representative of a small number of observations for most purposes, 
yet, a readily achievable number in a test environment. Fifty runs of 
100 impacts each were conducted sequentially so that a total of 5000 
random data points were drawn from the 1244 impacts available. As a test 
of the randomness of the samples, the mean radial error for each of the 
50 runs were compared and no two were found equal when rounded to the 
nearest integer value. 

The comparisons were conducted for r^ values of 30 through 80 feet in 
10 foot increments. The interval length (w) chosen for the multinomial 
sampling distribution was 10 feet. The a (probability of target destruc- 
tion) was chosen to be 0.9S. The CEP chosen for the Bayes prior was of a 
nominal value and was over 15 feet different from the value observed in 
the 1244 impacts. 

Figure 5 presents a plot of a versus r^ for the different measurement 
methodologies. Qualitatively, the circular normal yields the worst a 
values and the Bayesian method the best. The Bayesian method completely 
dominates for r^ values of 50 feet and greater. All the estimates approach 
a common value at r^ = 70 feet. An interesting point is that the Cauchy 
distribution appears to be a better estimator than the Cauchy plus direct 
hits distribution. 

The average A values for r^ values equal to and greater than 30 feet 

are; 
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Comparator Values Using A-7E 
Data. 
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1. Normal plus direct 

2. Bayesian 3.4 

3. Cauchy 3.6 

4. Cauchy plus direct 4.8 

5. Normal 7.9 



The slight advantage of the normal plus direct distribution results from 
the large difference at r^ = 30 feet. It can be concluded that the exact 
knowledge of the percent of direct hits was of considerable aid to the 
normal plus direct distribution. This is substantiated by the fact that 
the A value for the normal plus direct is smaller at r^ = 20 feet than 
at r =30 feet. Removing the r^ value of 30 feet results in both the 
Bayesian and Cauchy moving ahead of the normal plus direct. 

Though not presented in figure 5, several other techniques for calcu- 
lating the multinomial parameters (p^) were compared to the Bayesian. 

The maximum likelihood estimator of p^ = x^ / N and several weighted 
averages of three and five adjacent intervals (i.e., 



. «i,2 Pi-2 ^ «i.i P,-.i * P1 + Pjt1 ^ «H2 fu-z 



"i-2 * “i-1 ”l * “i+1 ”i+2 



yielded a values significantly greater than those resulting from the 
Bayesian technique. 



B. SIMULATION 

In order to test the Bayesian technique across a wider set of possible 
impact distributions, the desired distributions were simulated. The 
programs used to generate the distributions were those presented in 
Ref. 10 except for the Cauchy. The Cauchy was generated by solving 
equation (5) for r while using a random number generator to assign values 
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between 0 and 1 to F(r). A CEP of 150 feet and three percent direct hits 
were used in the generation of data points. The distributions simulated 
were the regular and the regular plus direct hit forms of the exponential, 
circular normal, circular Cauchy and uniform and a mixture of two 
circular normal distributions. The mixture consisted of 30 percent with 
CEP = 60 feet and 70 percent with CEP = 200 feet. The simulation 
routines were verified by plotting and by comparing the generated 
statistics to the input values. 

The same techniques were used for comparison as with the A-7E data 
(section IV. A). The percent of direct hits was assigned the known value 
of three percent. The r^ value chosen was 50 feet and the Bayes prior 
was CEP = 150 feet. One hundred runs of 100 samples each were conducted. 

Figure 6 presents the A value for each of the techniques and each of 
the simulated distributions. The Bayesian technique provided the best 
estimator for the exponential, uniform and mixed normal distributions. 
Surprisingly, the Bayesian technique also provided the best estimator for 
the Cauchy plus direct hit distribution but by a very small margin. 

Table I presents the same results in terms of the ranking of each 
technique for each distribution. The Bayesian was best overall. 

To ensure that the test was not biased by the use of the exponential 
and uniform distributions, these were removed. Table I also presents 
these results and the Bayesian technique is still seen to be the best. 
Interestingly, both the Cauchy and normal plus direct surpass the Cauchy 
plus direct when the exponential and uniform distributions are removed. 

It can be concluded that the Bayesian technique provides the most 
exact estimator of the techniques tested over a wide range of possible 
impact distributions. 
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# Bayes 
O Normal 
X Normal 
O Cauchy 
X Cauchy 



Distribution 

Figure 6. 

Comparator Values for Simulation Data. 
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Ranking of Comparator Values for Simulation Data 
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C. SENSITIVITY 

The A-7E data was used to test the sensitivity of the CEP value used 

for the Bayesian prior. As before, 50 runs of 100 samples each were 

conducted. The r„ value used was 50 feet. CEP values from 120 to 180 
e 

feet in increments of 10 feet were tested. 

Figure 7 presents the A values versus the CEP values. The A values 
range from 2.1 to 3.1. The minimum A value does not occur at the true 
CEP. The A values at r^ = 50 feet, for three of the other techniques 
are presented for comparison. 

It can be concluded that the Bayesian technique is relatively insensi- 
tive to the prior CEP value chosen when 100 data points are available, and 
the prior CEP value is reasonably close to the true CEP value. 
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Figure 7. 

Sensitivity of Bayes Prior A— 7E Data. 
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V. COST-EFFECTIVENESS GUARANTEE 



The purpose of this section is to attempt to employ the methodology 
of cost-effectiveness analysis in specifying and evaluating a contract 
guarantee for an A/G delivery system. Cost-effectiveness analysis has 
been defined as "a method for studying how to make the best of several 
choices. Cost-effectiveness is always used in relation to the effective- 
ness of alternative systems, organizations or activities." [11, p. 1] It 
is apparent that comparison of alternatives is the core of cost- 
effectiveness as applied to choice theory. If its techniques are applied 
to a single system, caution must be exercised. 

It can be seen from section I.B.l that present specifications some- 
times guarantee accuracy in a restrictive sense which inhibits both the 
guarantee and the evaluation to determine specification compliance. It 
is also notable that most guarantees avoid the issue of costs except for 
penalty values. 

The cost-effectiveness approach has proven valuable in the choice of 
a system from a set of alternatives and could prove valuable in guarantee- 
ing a particular system. Additionally, the cost-effectiveness based 
guarantee provides a stepping-stone toward controlling cost overruns. 

The MOE (1/n) required in a cost-effectiveness analysis has been 
presented in section II.B. 

A. COSTS 

1 . Types of Cost 

The three types of total or life cycle costs are research and 
development (R&D), investment and operating costs. To fully account for 
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the "cost" of a system, these must be expressed as "economic costs" or 
benefits lost due to the use of the resources required to develop, procure 
and operate a particular system. Thus, these costs include hardware, 
manpower, new facilities, supplies, dollar, etc.; everything directly 
related to the decision to achieve the system [12, p. 25,66-67]. 

Care must be taken when costing an A/G delivery system that is 
to be installed in an airframe so that the costs do not include those that 
are incurred by the airframe independent of the A/G system [13, p. 5]. 

Also the concept of sunk costs [12, p. 33] must be considered if some of 
the components of the system are currently developed or procured. These 
costs should not be included. 

Another cost that is sometimes erroneously included in computing 
system costs is the attrition of systems due to combat or operational type 
losses. Once the system is developed and procured, the only pertinent 
cost is the operating cost. 

The cost of the weapons expended to achieve the MOE are not 
included due to their negligible effect compared to the other costs. The 
MOE qualitatively reflects the increased weapons cost of an inefficient 
system. 

The methods of measuring the costs are well detailed by Fisher 
[12] and many other authors and need not be repeated here. However, the 
total cost concept has serious connotations in regard to a guarantee that 
will be fair for many different systems and needs to be further discussed. 

2. Total Cost 

Total costs represent the total resource impact or full economic 
cost of the system. Necessarily, the magnitude of these costs is highly 
dependent on the number of systems purchased and the operating lifetime 
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of the system. (The buy size and lifetime are normally estimated during 
the conceptual phase of a system and the determination of these values is 
not essential to this thesis.) If these total costs are used to determine 
a cost-effectiveness guarantee, systems with a large buy and/or a long 
lifetime will be unfairly penalized by the requirement for a higher 
effectiveness level than a similar (in performance) system of which only 
a few are purchased and have a shorter lifetime. The logical conclusion 
is that some form of normalization is in order. 

Varying opinions exist as to whether or not costs should be 
normalized [6, p. 40 and 11, p. 29]. In general, normalization tends to 
hide what the total cost is and in the usual context of cost-effectiveness 
analysis it is important to be fully aware of the total cost. However, in 
the proposed context, the lack of normalization creates the inequity shown 
above, Thus, normalization is considered applicable in the guarantee 
context. 

3, Cost Model 

Close examination of the three incremental costs shows that: 

a. The R&D cost is nearly independent of the buy size and system lifetime. 
Therefore, this cost can be used with no normalization. 

b. The investment cost is most dependent on the buy size and it seems 
natural to normalize it over the number of systems resulting in units of 
dollars per system. 

c. The operating cost is dependent on both lifetime and buy size. 

Normalizing over both yields units of dollars per unit of time per system. 

This will be expressed as an annual operating cost per system in this thesis. 

Now, there are three different costs with three different dimen- 

2 4 

sions. The cost-effectiveness model has been transformed from E to E 
(where E*^ is Euclidean n-dimensional space). 
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The normalization of the costs in the cost model is unique to 
the guarantee context and these costs should be used with care in other 
contexts . 

B. COST-EFFECTIVENESS RELATIONSHIP 

The various techniques that could possibly be used to determine the 
functional relationship between effectiveness and cost are detailed in 
many sources such as Theil [14] and Raiffa and Schlaifer [17]. Prior to 
the use of any of these techniques, however, a data base is required. 

1 . Data Base 

A proper, though not extensive, data base exists for determining 
the functional relationship. The cost data, reliability data, and weapon 
delivery accuracy data for present and past systems is historical and can 
be used [15, p. 11-13]. 

The data available needs to be transformed into a form compatible 
with the cost-effectiveness framework outlined above. The availability, 
dependability (based on a standard mission time) and capability are used 
to compute 1/n. The cost data needs to be partitioned in the appropriate 
accounts; i.e., R&D, investment per system, and annual operating per 
system. Other adjustments to the data may be necessary. Examples of 
these are well detailed in a RAND report [15, p. 17-32]. 

2. Technological Dynamics 

With the data transformed into the proper framework, two other 
aspects requiring attention still exist. First, is the fact that the 
technology under which each of the systems was produced may not be the 
same. The natural assumption is that the technology is increasing 
chronologically and that later systems are more effective. Increased 
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technology has historically carried an increased price tag. This is 

partially what the cost-effectiveness curve is representing. However, 

the possibility of cost decreasing, technological break-throughs is real. 

If during the determination of the functional relationship, one or more 

data points appear not to fit the others then these must be studied 

carefully to see if such a break-through did exist. When this is the 

case, a new cost-effectiveness frontier should be generated based on the 

new technology. Generally, omitting the data points representative of 

the old technology will be sufficient. It may be though, that there are 

too few data points from the new technology. In this case, the shape of 

the frontier could be determined using the old technology and the 
/ 4 V 

"height" (in an E sense) from the new technology. 

3. Inflation 

The second aspect is that the dollar used to cost a particular 
system is not the same dollar used to cost chronologically future or prior 
systems. This is not serious though as well developed techniques exist 
[15, p. 23-32] to account for the time dependent value of the dollar. All 
that needs to be done is to select a date for the base value of the dollar 
and transform all dollar values to this base date. It may be desirable 
to change the base date periodically to keep it fairly close to the 
present. 

4. Functional Relationship Model 

The statistical technique used to determine the functional 
relationship between cost and effectiveness needs to include the ability 
to explicitly state the uncertainty incurred in the predicted form. A 
Bayesian regression technique might be reasonable, due to the ability to 
make probability statements concerning the estimated parameters but no 
extensive study of this has been conducted by the author. 
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The successful completion of the above steps will result in 
the depiction (mathematically) of a hypersurface in that represents 
a cost-effectiveness frontier. 

5. Uncertainty Considerations 

When estimating a hypersurface of unknown form from a few data 
points, there is a high degree of uncertainty. Though it would be 
appealing to use the generated hypersurface for the guarantee, it might 
be unfair. Thus, the prediction uncertainty is required. The direction 
of the prediction error is also uncertain and in fairness the applicable 
lower bound of the prediction error interval should be used. 

C. GUARANTEE 

4 

The cost-effectiveness guarantee is in E and cannot be depicted 
graphically. However, an interpretation of the concept can be portrayed 
using a representative cost axis as shown in Figure 8. 

Adding a minimum effectiveness and maximum cost to the guarantee 
defines the areas as shown in Figure 8 including the feasible region. 

The minimum effectiveness requirement is straight forward but the maximum 
cost has hidden implications. In order to guarantee a maximum total cost, 
that total cost must be apportioned among the modified incremental costs. 
Thus, though specifying a maximum total cost is preferable, it may be 
difficult. The alternative is to specify maximum incremental costs 
where the sum is not the desired maximum total cost. 

The guarantee also must specify the mission time from takeoff to 
on target and the r^ value. If these values are altered, the entire cost- 
effectiveness frontier must be regenerated. Therefore, these values must 
be chosen wisely. 
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(R8iD, Inv/Sys, Annual Op/Sys) 
Figure 8. 

Typical Cost-Effectiveness Guarantee. 
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1 . Penalty Assessment 

The historical method of assessing penalties is to assign a dollar 
penalty for each incremental unit outside the guaranteed value. In theory 
this penalty assessment seems acceptable but in practice it often happens 
that the penalty costs, when assessed, are charged back to the procuring 
agency disguised as increased overhead, ground support equipment or other 
type costs adding to the total system cost. The theory of a cost- 
effectiveness guarantee would eliminate this practice but the method of 
penalty assessment could also eliminate it. The manufacturer could be 
required to meet the guarantee by reducing cost or increasing effectiveness 

2. Guarantee Currency 

A periodic review of the guarantee would be necessary to keep it 
current. A logical mechanism for ensuring that the review is accomplished 
would be to require a regeneration of the cost-effectiveness hypersurface 
each time new actual data becomes available and require a constructive 
review each time the guarantee is used. The minimum effectiveness and 
maximum cost portion of the guarantee could be unique to each system and 
these parameters could be defined during the latter type review. 

3. Guarantee Parameters 

The parameters for the cost-effectiveness guarantee are summarized 
as: 

a. Minimum effectiveness 

b. Maximum cost of 

1) R&D 

2) investment per system 

3) annual operating per system 

c. Cost-effectiveness hypersurface 

d. Penalty assessment 
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e. Mission time from launch to target 

f. r^ value 

e 

Other parameters peculiar to some system or overlooked herein might 
also require specification. 
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APPENDIX A 



DERIVATION OF THE ACCURACY ESTItWORS 

The detailed derivation of the accuracy estimators, based on Bayesian 
techniques, is presented below. 

A. LOSS FUNCTION 

The concept of a Bayesian estimator is minimization of the expected 
loss; thus, a loss function is needed. 

A loss function is a function that assigns a number (called the loss) 
to each combination of decision and state of nature [8, p. 122]. In the 
problem at hand, the state of nature is the theoretic number of targets 
destroyed per sortie (1/n). The decision is the estimate of 1/n, symbol- 
ized as 1/n. Any nonnegative function of the error (1/n - 1/n) can be 
used as a loss function for estimation [8, p. 226]. The most commonly 
used are absolute value of the difference and squared error (quadratic). 

DeGroot shows [8, p. 227-228] that the quadratic loss is an acceptable 
approximation for a wide range of nonnegative loss functions. Principally 
because of this, the quadratic loss was chosen. An implicit assumption 
involved in this choice is that the disutility of an overestimate is 
identical to that of an underestimate. 

The loss function is 

(A.l) L = a (1/n - 1/n) where a is some nonnegative constant. 
Recalling equation (15) (from section II .C) with = 1 , the loss can be 
expressed as: 

(A. 2) L = — ^ [In(l-p) - In(l-p)]^ where p is the estimator of p 

In^(l-a) 
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Letting the decision be D = In(l-p) and the state of nature be 
0 = In(l-p), L = K(D-e)^ where K = a/ln^(l-a). DeGroot shows [8, p. 228] 

4 

that, for this loss function, the Bayes decision against any given distri- 
bution of 0 is ; 

(A.3) D* = E[e] 



B. SAMPLING DISTRIBUTION 

One of the attributes listed in section I.D was that the methodology 
be distribution free. The use of a multinomial distribution to depict 
the impact pattern achieves this goal. The form of the multinomial 
probability mass function: 



(A. 4) f Cx-| , . . . ,X|^ 1 N ,p-| , . . . ,p ) 



k 

where E x. 
i=l ^ 

k 



N 

1 






p^ ^ 0 i=l,,..,k 



permits the cumulative distribution curve to take nearly any shape from 

convex to concave and many conbi nations thereof. 

Due to the desirable independence from run-in heading shown in section 

II. E, the radial miss distance was chosen as the variable for the density. 

Figure A.l relates the geometry of the impact pattern to the multi- 

t h 

nomial density. The p^. are the probability of an impact in the i^” 

J. L 

interval and the x^. are the number of impacts in the i^^ interval. 
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Figure A.l 

Relationship of Multinomial Density to Target Geometry. 
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The length of the intervals (w) can be assigned as desirable. They 

need not be of uniform length, though it is assumed here that all but 
t h t h 

the will be. The interval will include the portion of the real 

th 

line beyond the end of the (k-1)'^^ interval. It would seem logical to 
choose an interval length that is some function of the measurement 
accuracy of the data collection method. A reasonable choice seems to be 
an interval length equal to twice the measurement error in the data 
collection. The data analyzed for this thesis was collected with a 
measurement error of t5 feet leading to w = 10 feet. 

Another consideration is the number of intervals (k). The choice of 
k should be based on the number of data points available for analysis, 
the interval length and the maximal miss distance of interest. The observed 
distribution of impacts may also influence the choices of k and w. In 
general, a study of the system, observed data, and data collection method 
should enable one to assign reasonable values to k and w. 



C. CONJUGATE FAMILY 



The use of a conjugate family in Bayesian analysis simplifies the 
mathematical manipulation and ensures an estimate based on a sufficient 
statistic [8, p. 159]. 

The conjugate family for the multinomial is the multivariate Beta 
or Dirichlet distribution [8, p. 174]. The functional form of the 
Dirichlet density is: 



(A.5) f(p^, 

where 





k 

z p. = 1 

i=l ^ 



r(g^+ ... + 1 

r(a^) ••• (a^) ^ . 



> 0 i=l,...,k 

g* ^ 0 i^lj.f.jk 
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The expected value and variance are: 



k 



(A.6) EEpJ 



a 




0 




DeGroot shows [8, p. 174] that if the prior parameters are a^,...,a|^ 
and are the number of observations in each interval, then the 

posterior parameters are a^+x-i 

D. ESTIMATORS 

It was stated in paragraph A (Loss Function) that the Bayes estimator, 
D*, was equal to the expected value of 0 = In(l-p). There is an obvious 
difference between the distribution of 0 and the posterior distribution 
of the vector of p^'s. One assumption and one approximation are used to 
derive the Bayes estimators for the p^. 's. 

The assumption involves the relation p = A^DC. C is the probability 
of an impact within r^ of the target given some state of the system. 

Thus, C is related to the posterior of the p^'s by: 



m 

(A. 8) C = E p. where m is the smallest integer satisfying the 

i=l ^ 

inequality m ^ r^/w 



Note that E[C] = E[ Z p.] = Z E[p.] 

i=l ^ i=l ^ 



m m 



i=l 



m 

z 




Assuming the reliability of the system is unity (A^D = 1) permits setting 



E[p] = E[C]. 
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The approximation involves the relation 0 = In(l-p). For reasonable 
values of r^, the values of p are relatively small (less than .2). For 
such values of p, ln(l*p) can be approximated by p with a maximum error 
of approximately 15 percent. 

The approximation p = In(l-p) permits setting D* = E[e] = E[p] = E[C] 
Thus, the Bayes estimators for the p^ are: 

a. + X. 

(A. 9) p.* = - and 

. m a. + X. 

(A. 10) p* = E -4—^ 
i=l 0 

E\ PRIOR PARAMETERS 

For the subject problem several schemes for assigning the prior 
parameters were possible. The prime factor dictating the choice was to 
require a minimal number of parameter values to be assigned. 

One of the most widely used and well known accuracy parameters is the 
CEP. The CEP is also one of the easiest parameters to assign a priori. 

The assignment can be based on system simulation, system design or some 
other method tempered by experience. The procedure for using CEP as a 
single assignment parameter, as discussed below, is Ad Hoc. Several 
assumptions are made, the justification of which is made in section III.B. 

In terms of the p-'s, the CEP is defined as: 

0 

.5 = E p. where j = CEP/w. 
i=l ’ 

Assuming E[p^J are equal for t = l,,..,j and a uniform length (w) for 
intervals 1 , . . . ,j : 
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• 5/ J > i 



eCp^] = 



and from equation (A. 6), the are equal for i = 1 j. Allowing the 

interval to be the remainder of the distribution, E[P|^] = .5. 

From equation (A. 6) and the discussion above, 

j 

“o = ‘"i * “k ' * “k 



and E[p.] = ' 

* “k 



thus , 

a. = (ja^ i = 

and 

a|^ = *5 (ja^. + aj^) 



These equations are solved to find 

aj^ = ja^ ■ i=l,...,j as might have been expected. 

Attempting to use the derived values and assigning values to the 
variances (equation (A. 7)) leads to an inconsistent system of equations. 
This can be resolved by assuming that each p^. is distributed symmetrically 
about its expected value. Then the variance of the p^. for i=l,...,j can 
be set such that 95 percent of the probable p^ will be nonnegative. It 
follows that; 

V[p.] = (E[p.]/1.64)^ 

For i=l ,. . . ,j ; 

V[p.] = (l/3.28j)2 



57 



Solving for the yields; 



(A.ll) 



“i 



2.69j - .845 

j 



,j 



and 



(A. 12) a,^ = ja. = 2.69j - ,845 



It has been shown, with several assumptions, that the prior parameters 
can be derived given the prior CEP. 
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APPENDIX B 



SAMPLE DERIVATION OF 



The procedures for using the Bayesian technique suggested in this 
thesis are illustrated by a simple, fictitious example involving a sample 
of 10 impacts. Suppose the observed radial miss distances (in feet) for 
dive maneuvers in the prime mode are: 

7, 92, 56, 37, 23, 6, 88, 75, 29, 41 
An interval length of 10 feet and prior CEP of 50 feet will be used. 



j = CEP/w = 50/10 = 5 



thus k = 6 



a = 2 , -69(5) - .845 ^ 2.52 i=l,...,5 

* 5 



ag = 12,61 



The values of interest can be tabulated as follows: 



i 


interval 

endpoints 




X.* 

1 


«i + X. 


p.** 


1 


0-10 


2.52 


2 


4.52 


.13 


2 


10-20 


2.52 


0 


2.52 


.07 


3 


20-30 


2.52 


2 


4.52 


.13 


4 


30-40 


2.52 


1 


3.52 


.10 


5 


40-50 


2.52 


1 


3.52 


.10 


6 


50-«> 


12.61 


4 


16.61 


.47 



★ 



.X. 






is the number 

X, 



“i ^ 



1 



of impacts observed in the 



6 

E + X.) . 

i=l ^ ‘ 



i^^ interval. 
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APPENDIX C 



EXAMPLE DERIVATION OF p 

The equation p = A^DC given in the WSEIAC report [7] is explained 
by the following fictitious example using the C vector component as 
derived in Appendix B. 

Consider a system which has a prime mode and one degraded mode. The 
possible system states are defined as; 

A - prime mode 
B “ degraded mode 
C - inoperative 

Suppose the availabilities (probability that the system is in some 
state at the beginning of the mission) are; 

P{A> = 1/2 

P{B} = 1/4 

P{C} = 1/4 

Thus, A^ = Cl/2,1/4,1/4). 

Assume the system cannot be repaired in flight. The dependabilities 
(probability that the system is in some state at a specified time after 
takeoff given the system state at the beginning of the mission) are 
assumed to be: 

Given state A at the beginning of the mission*, 

P{A} = 1/2 

P{B} = 1/4 

P{C} = 1/4 

Given state B at the beginning of the mission*. 
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P{A} = 

P{B} = 3/4 

P{C} = 1/4 

Given state C at the beginning of the mission; 
P{A} = P{B} = 0 

P{C} = 1 



Thus, D = 



1/2 1/4 1/4 

0 3/4 1/4 

0 0 1 



and A^D = 0/4,5/16,7/16) 



The A^D is constant for a constant mission time and in most systems 
will probably remain fairly constant over various mission times. There- 
fore, in the weaponeering usage this could be accepted as a set of values 
unique to each A/G delivery system. Then, only the final calculation 
(shown next) need be performed for various r^ values. 

The capabilities (probability that weapon impacts within r^ of the 
target given the system state at the time of release) are as derived in 
Appendix B using r^ = 30 feet^ : 

3 . 

P{hit|A} = E p. = .33 = 1/3 

i=l ^ 

P{hit|B} = 1/6 

P{hit|C} = 0 

Thus, = (1/3, 1/6,0) 



Hhe P{hit|B} is assumed to be 1/6. 
was not computed in Appendix B. 



The value for the degraded mode 
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1 




Solving for p = A DC 



p = (1/4,5/16,7/16) 



1/3 

1/6 



j 



= 13/96 



Therefore, the estimated probability of placing a weapon within 30 feet 
of the target with this system is 13/96. 
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APPENDIX D 



WEAPONEERING EXAMPLE 

The weaponeering usage of the proposed measurement and methodology 

A 

will be illustrated using the fictitious p calculated in Appendices B 
and C. 

For the computed p = 13/96 and assuming a = .95, n can be calculated 
as: 

n = In(l-g) ^ ln(.05) ^ 21 ' 

In(l-p) lnC83/96) 

A graph similar to that shown in Figure D.l could be used, vice 

/\ A 

solving the above equation, to find n. Entering the graph with p (called 

A 

the cumulative percentage) and a, the n could be found. 

In general usage, the weaponeering requirements would be: 

1) Capability tables for the applicable system and degradation modes. 

2) A^D vector for the applicable system. 

3) Graph similar to Figure D.l. 

If the A^D were negligibly dependent on mission time, requirements 1 and 

2 could be combined into a single table of p values for the various values 

of r^. 
e 

At the ship or airwing level, the weaponeer could use the individual 
squadron capability tables or an airwing table aggregated over particular 
systems in the airwing, (i.e., A-7 capability, F-4 capability, A-6 
capability). In mixed strikes, the ship level weaponeer could use a 
ratio weighting factor for the p based on the scheduled mix. As an 
example consider a strike consisting of 8 A-7, 4 A-6 and 8 F-4. The p 
value for the strike could be p* = ^ '^*^F-4* 
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Required Sorties to Achieve Target Destruction (n) 



80rQ= -99 




Cumulative Percentage (Cj) 
Figure D. I 
Weaponeering Graph 
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